2026-02-05
toy exampleToday’s class involves a walk-through of the toy example, which is a simple simulated observational study of a treatment on three outcomes (one quantitative, one binary, and one time-to-event) which we will use to demonstrate the completion of 13 tasks using R, which include:
toy exampleNote we have three other (more realistic) examples we’ll share in time: lindner, dm2200 and rhc.
toy exampleThe toy example presents methods for building and using propensity scores with simple simulated data.
toy exampletoy example (and a break)We determined whether adherence to recommendations for coronary angiography more than 12 h after symptom onset but prior to hospital discharge after acute myocardial infarction (AMI) resulted in better survival. Using propensity scores, we created a matched retrospective sample of 19,568 Medicare patients hospitalized with AMI during 1994–1995 in the United States. Twenty-nine percent, 36%, and 34% of patients were judged necessary, appropriate, or uncertain, respectively, for angiography while 60% of those judged necessary received the procedure during the hospitalization. The 3-year survival benefit was largest for patients rated necessary [mean survival difference (95% CI): 17.6% (15.1, 20.1)] and smallest for those rated uncertain [8.8% (6.8, 10.7)]. Angiography recommendations appear to select patients who are likely to benefit from the procedure and the consequent interventions. Because of the magnitude of the benefit and of the number of patients involved, steps should be taken to replicate these findings.
Because we collected detailed clinical information describing admission severity of the patient and characteristics of the hospital to which the patient was admitted, we assumed that treatment (angiography vs. no angiography) was randomly assigned with probabilities that depended on the observed covariates alone.
We then employed a propensity score approach to compare survival between those receiving angiography (“cathed”) and those who did not (“not cathed”) within each category of appropriateness. The propensity score is a measure of the likelihood that a patient would have undergone angiography using the patient’s covariate scores.
To estimate the propensity scores, we fitted a logistic regression model in which the outcome was the log-odds of undergoing angiography more than 12 h after symptom onset but prior to discharge.
{The covariates used in the propensity score} consisted of patient (demographic, comorbidity, admission severity) and hospital characteristics as well as interactions among the covariates.
We assumed that missing observations were missing at random, implying that the mechanism by which data were missing is unrelated to information not contained in our observed data. For discrete-valued variables, we included a binary variable that represented “missing.” In the case of continuous-valued variables, we created two variables: a binary variable indicating whether the variable was measured and if measured, a continuous variable indicating the value of the variable.
Once the model was estimated, we stratified the cohort by clinical indication, and within an indication, matched each patient who underwent angiography to a patient with closest estimated propensity score who did not. We included in our analyses only those matches that were within 0.60 of the pooled standard error of q(X) where q(X) is the estimated logit. This method of defining the closeness of a match is referred to caliper matching and is the observational study analogue of randomization in a clinical trial.
Fig. 1 (next two slides) summarizes our methods for identifying and creating the matched sample.
We matched 57% of the 17,304 cathed patients to noncathed patients using estimated propensity scores.
The unmatched angiography patients were more likely to be admitted to large, teaching, urban hospitals with the capability to perform invasive cardiac procedures; were younger; were less sick; and had less comorbid disease compared to the angiography patients for whom we found matches. Prior to matching, the average predicted propensities to undergo angiography were 65% and 30% in the two groups; after matching, the propensities were within 4 percentage points.
The propensity score approach, a technique that has been employed in other recent medical studies, reduces the collection of many confounding variables to a single variable that permits easy comparisons of group differences. Although we were successful in reducing the bias that may have resulted from inexact matching on observed covariates, we were only able to adequately match 57% of all patients who underwent angiography. The unmatched angiography patients were generally younger and healthier than the matched angiography patients and if included in the comparisons would have biased the effect of angiography towards a larger benefit.
Although the exclusion of the unmatched patients may have introduced a bias, their inclusion would have also compromised the comparability of the final matched groups. Because it is difficult to completely rule out all these biases, it is important for others to validate our findings.
In conclusion, coronary angiography following AMI was associated with increased survival for a relatively contemporary cohort of Medicare beneficiaries who had an AMI. The benefit was present in all categories of appropriateness that applied to these patients. Because of the magnitude of the benefit, the recent experiences of the patients, and the size of the group involved, the data suggest that not only is underuse of this procedure after AMI prevalent but may explain the lack of long-term survival differences between high-use regions and low-use regions. Because we were unable to match all patients who underwent coronary angiography, research should be undertaken to replicate our findings.
Sensitivity to Unmeasured Covariates
Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (Connors et al. 1996)
Prior (small) observational studies comparing RHC to non-RHC patients:
Big Problem: Selection Bias. Physicians (mostly) decide who gets RHC and who doesn’t.
Why not a RCT?
Panel (7 specialists in clinical care) specified important variables related to the decision to use or not use a RHC.
RHC patients were more likely to
RHC patients were less likely to
RHC patients had
Reweight each treated patient by 1/PS, and each control patient by 1/(1-PS).
Results of this Weighting Approach on the next slide…
500 Class 4 | 2026-02-05 | https://thomaselove.github.io/500-2026/